Multi-band imaging sensors typically are designed such that each band of the imaging system is sensitive to a pass-band of electromagnetic radiation. For example, a standard color imaging system consists of three bands (or arrays of detectors) sensitive to red, green, and blue light, respectively. Imaging systems such as multi-spectral or hyper-spectral systems contain many detector bands. These systems may contain spectral bands sensitive to non-visible parts of the electromagnetic spectrum such as to NIR (near-infrared), SWIR (short-wave infrared) or MWIR (mid-wave infrared) in addition to bands sensitive to red, green, and blue light. Color composite imagery is commonly formed from multi-spectral imagery and hyper-spectral imagery by mapping three selected bands to the red, green, and blue bands of an output display device such as a video monitor, or a digital color printer.
Every detector of a given spectral band in an electronic image sensor, such as a CCD image sensor, may have a different response function that relates the input intensity of light (or other electromagnetic radiation) to a pixel value in the digital image. This response function can change with time or operating temperature. Image sensors are calibrated such that each detector in a given spectral band, has the same response for the same input intensity (illumination radiance). The calibration is generally performed by illuminating each detector of the spectral band with a given radiance from a calibration lamp and then recording the signal from each detector to estimate the response function of the detector. The response function for each detector is used to equalize the output of all of the detectors such that a uniform illumination across all of the detectors will produce a uniform output. This calibration is typically performed separately for each band of a multi-band imaging system.
FIG. 1 shows a schematic of an image acquired by a linear image sensing arrays. In such an image, if errors in estimating the response curve of a detector are different from the errors in the response curve of an adjacent detector, the detector responses will not be equalized and streaking 2 will appear in the image along the scan direction indicated by arrow A. Since these calibration errors can occur within each band of a spectral imaging system, and several bands of a spectral imaging system are often combined together to form a color composite image, the streaking 2 may appear in various colors. Often to achieve a very long array, several image sensor chips are joined together to form a single linear image sensor. Slight differences in response between chips (due to variations in sensor chip manufacturing, or sensor electronics processing) can lead to large calibration errors between chips. When calibration errors occur between chips, the streaking is generally referred to as banding, as illustrated at reference numeral 4.
Even when the detectors are calibrated to minimize the streaking in the image, some errors from the calibration process are unavoidable. Typically, a spectral filter is placed on a given detector, or sensor chip to create an imaging band sensitive to a specific region in the electromagnetic spectrum. Depending on the architecture of the sensor array, it may be necessary to have several spectral filters of the same bandpass to cover the entire array. Often, due to the variations in the spectral filter manufacturing process, the filters that are placed over the detectors in a given band may be slightly different in spectral bandpass and spectral shape. In addition, material variations, and the angle of incidence of light on a spectral filter, causes additional spectral variations depending on the position of the spectral filter on the sensor array. As a result, each detector within a spectral band is sensitive to a slightly different spectrum of light, but they are all calibrated using the same calibration lamp with a broad, non-uniform spectrum. Since the scene spectrum is unknown, the calibration process assumes that the spectrum of the calibration lamp and scene are identical. The spectrum of the calibration lamp will usually be somewhat different than the spectrum of the scene being imaged, hence calibration errors will occur. This calibration error is also referred to as spectral banding. Calibration errors also occur because the calibration process includes an incomplete model of the complete optical process and because the response function for each detector changes over time and operating temperature.
Streaking can be seen in uniform areas of an image acquired by a linear detector and become very apparent when the contrast of the image is enhanced. Calibration differences between the red, green, and blue detectors of color imaging systems (or any of the bands in a multi-spectral or hyper-spectral imaging system) produce streaks of varying colors in the composite color image. These streaks not only reduce the aesthetic quality of digital images but can impact the interpretability of features in the images. Streaking also severely degrades the performance of pattern recognition, feature extraction algorithms, image classification algorithms and automated or semi-automated target recognition algorithms.
Streaks can be attenuated by reducing the contrast of each image band or by blurring each image band in a direction perpendicular to the streaking, but these methods degrade the quality of the overall image. Previously developed algorithms designed to remove streaks from digital imagery while preserving the sharpness and contrast of the image were designed to remove streaks on single band imagery; not on multi-band imagery. These algorithms only take into account spatial information present in the image to remove streaks. No attempt is made to examine additional color information or spectral correlation available in multi-spectral or hyper-spectral imagery to remove streaks. As a result, when applying these techniques to multi-band imagery, these algorithms do not completely remove all of the color streaks present in the original image and may introduce objectionable color streaks or bands as artifacts. These algorithms do not preserve and/or restore the overall color fidelity of the image. In addition, applying algorithms designed to remove streaks from single band imagery on color or multi-spectral imagery, is a non-optimal method for streak removal for color imagery or multi-spectral imagery, as these algorithms do not use all of the available information that is present in multi-band imagery during the streak removal process.
U.S. Pat. No. 5,065,444, issued Nov. 12, 1991, to Garber discloses a method of removing streaks from single band digital images by assuming that pixels in a predetermined region are strongly correlated, examining the pixels in the region, computing the difference between the pixels in the region, thresholding the pixel differences lower than a predetermined value, computing a gain and offset value from the distribution of differences, and using the gain and offset value to remove the streaking. Methods that assume a strong correlation between pixels that are near each other, such as the one disclosed by Garber will interpret scene variations as streaks and produce additional streaking artifacts in the image as a result of attempting to remove existing streaks. FIG. 2a shows an image having streaks 2 and linear features 6 that are in the same direction as the streaks. As shown in FIG. 2b, the correction of the streaks 2 using the method taught by Garber removes the streaks 2, but results in additional streaking artifacts 8. Applying the method by Garber to each spectral band of multi-band images will result in objectionable color streaks and color banding artifacts.
U.S. Pat. No. 5,881,182, issued Mar. 9, 1999, to Fiete et al., which is incorporated herein by reference, discloses a method of removing streaks by comparing the means between a local window region of two columns of data in the imagery to determine if a streak was present, and presenting statistical methods to calculate a gain and offset to remove the streaks. To apply this method on spectral or color imagery (imagery that consists of more than one spectral band), this method would be applied independently to each band of the spectral imagery, and then the bands of the spectral imagery are recombined to form a color composite image. The method of Fiete et al. looks only at the spatial and luminance information within each band independently; hence calibration differences between each of the bands may not be corrected. As a result, when applied independently to each band of a multi-band image, and then combining these spectral bands together to form a color composite image, all of the color streaks may not be completely removed from the imagery. Yet, when the three spectral bands from a multi-band spectral image, each containing some unremoved streaks and slight banding artifacts, are combined to form a composite three band color image, these artifacts manifest to form objectionable color streaks and color banding in the color composite imagery. The method of Fiete et al. does not adequately remove color streaks from multi-band imagery.
There is a need therefore for an improved digital image processing method for removing streaks in color or multi-band images. The method presented here is an improvement of the method of Fiete et. al. to better remove color streaks and bands from multi-band imagery.